Graphic code display method, terminal and storage medium

ABSTRACT

A graphic code display method, a device, a terminal, and a storage medium are provided, belonging to the mobile payment field. An example method comprises: identifying an action, in an unlocked state, based on sensor data collected by a sensor; capturing an image by camera, in response to an identified pre-action, the pre-action being the action prior to displaying the graphic code to a code scanning device; detecting the code scanning device in the captured image to acquire detection result; and displaying the graphic code, in response to the detection result indicating that the image contains the code scanning device. A user only needs to show the pre-action for displaying a graphic code, and mobile terminal may automatically trigger to display the graphic code, without pre-opening an application and a graphic code display function, simplifying the graphic code display process.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of International Patent ApplicationNo. PCT/CN2021/094670, filed on May 19, 2021, which claims the priorityof Chinese Patent Application No. 202011615564.8, filed on Dec. 31,2020, both of which are herein incorporated by reference in theirentirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate to mobile paymenttechnology, and more particular, to a graphic code display method, aterminal, and a storage medium.

BACKGROUND

With the continuous development of mobile payment technology, more andmore users tend to use a mobile payment function of a terminal foroffline payment. Common mobile payment methods of a terminal includedisplaying a payment code and scanning the payment code.

When using a terminal to display the payment code for payment, the userfirst needs to open the application with the mobile payment function,and then enter the payment code display entry of the application, so asto display the payment code to the code scanning device to complete thepayment.

SUMMARY

Embodiments of the present disclosure provide a graphic code displaymethod,, a terminal and a storage medium. A described technical schemeis as follows.

In an aspect, an embodiment of the present disclosure provides a methodfor displaying a graphic code. The method comprises: identifying anaction, in an unlocked state, based on sensor data collected by asensor; capturing an image by a camera, in response to an identifiedpre-action, the pre-action being the action prior to displaying thegraphic code to a code scanning device; detecting the code scanningdevice in the captured image to acquire a detection result; anddisplaying the graphic code, in response to the detection resultindicating that the image contains the code scanning device.

In another aspect, an embodiment of the present disclosure provides aterminal. The terminal comprises: a processor and a memory; the memoryconfigured to store instructions which, when executed by the processor,cause the one or more processors to identify an action, in an unlockedstate, based on sensor data collected by a sensor; capture an image by acamera in response to an identified pre-action, wherein the pre-actionis the action prior to displaying the graphic code to a code scanningdevice; detect the code scanning device in the captured image to acquirea detection result; and display the graphic code, in response to thedetection result indicating that the image contains the code scanningdevice.

In another aspect, an embodiment of the present disclosure provides anon-transitory computer readable storage medium having storedinstructions that is executed by a processor of a terminal, cause theprocessor of a terminal to identify an action, in an unlocked state,based on sensor data collected by a sensor; capture an image by a camerain response to an identified pre-action, wherein the pre-action is theaction prior to displaying the graphic code to a code scanning device;detect the code scanning device in the captured image to acquire adetection result; and display the graphic code, in response to thedetection result indicating that the image contains the code scanningdevice.

Other features and aspects of the disclosed features will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, which illustrate, by way of example, thefeatures in accordance with embodiments of the disclosure. The summaryis not intended to limit the scope of any embodiments described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a principle diagram of a process for implementing agraphic code display method according to embodiments of the disclosure.

FIG. 2 illustrates a flowchart of a graphic code display methodaccording to an exemplary embodiment of the disclosure.

FIG. 3 illustrates another flowchart of a graphic code display methodaccording to an exemplary embodiment of the disclosure.

FIG. 4 illustrates a schematic diagram of a process for displaying thegraphic code according to an exemplary embodiment of the disclosure.

FIG. 5 illustrates another schematic diagram of a process for displayingthe graphic code according to an exemplary embodiment of the disclosure.

FIG. 6 illustrates another schematic diagram of a process for displayingthe graphic code according to an exemplary embodiment of the disclosure.

FIG. 7 illustrates another flowchart of a graphic code display methodaccording to an exemplary embodiment of the disclosure.

FIG. 8 illustrates another flowchart of a graphic code display methodaccording to an exemplary embodiment of the disclosure.

FIG. 9 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen fromupward to downward;

FIG. 10 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen fromupward to forward horizontal.

FIG. 11 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen fromupward to forward vertical.

FIG. 12 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen from backvertical to downward.

FIG. 13 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen from backvertical to forward horizontal.

FIG. 14 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen from backvertical to forward vertical.

FIG. 15 illustrates a schematic diagram of a change in gravitationalacceleration data, when in the process of flipping the screen from backvertical to upward.

FIG. 16 illustrates another schematic diagram of a process fordisplaying the graphic code according to an exemplary embodiment of thedisclosure.

FIG. 17 illustrates another schematic diagram of a process fordisplaying the graphic code according to an exemplary embodiment of thedisclosure.

FIG. 18 illustrates another flowchart of a graphic code display methodaccording to an exemplary embodiment of the disclosure.

FIG. 19 illustrates a schematic diagram of interface for switching thegraphic code process according to an exemplary embodiment of thedisclosure.

FIG. 20 illustrates a structural diagram of a graphic code displaydevice according to an embodiment of the disclosure.

FIG. 21 illustrates a structural diagram of a terminal according to anexemplary embodiment of the disclosure.

DETAILED DESCRIPTION

In order to make objectives, technical solutions and advantages of thepresent disclosure clearer, implementations of the present disclosurewill be further described in detail below in combination with thedrawings.

In the description of the present disclosure, “plurality” means two ormore. “And/or” describes an association relationship of associatedobjects, and means that there can be three kinds of relationships. Forexample, A and/or B may indicate that A exists alone, A and B exist atthe same time, or B exists alone. The character “/” generally indicatesthat the contextual objects are in an “or” relationship.

In the related art, when using a graphic code display function, the userfirst needs to open the application with the graphic code displayfunction, and then enter the graphic code display entry of theapplication to make the terminal display the graphic code, and thenpoint the graphic code to the code scanning device so that the codescanning device can scan the graphic code.

In order to simplify the process for displaying the graphic code and toachieve the graphic code display without the user’s perception (such aspayment without perception), as shown in FIG. 1 , an embodiment of thepresent disclosure provides a method for displaying a graphic code. Theterminal acquires the sensor data 102 collected by the sensor 101 duringthe working state, and identifies an action based on the sensor data102. When it is identified that the user has performed the pre-action103 for displaying the graphic code, the terminal further capturesimages by the camera 104 and detects the code scanning device in thecaptured image 105. When the image 105 is detected to contain the codescanning device 106, the terminal automatically displays the graphiccode 107.

During the entire graphic code display process, the user only needs toperform the pre-action of the graphic code display, and point theterminal screen to the code scanning device, and the terminalautomatically triggers the display of the graphic code, without the needfor the user to open the application and enter the graphic code displayentry, which simplifies the graphic code display process. The graphiccode display process will be described in detail below using exemplaryembodiments.

Please refer to FIG. 2 , which illustrates a flowchart of a graphic codedisplay method provided by an exemplary embodiment of the presentdisclosure. The method includes blocks as follows.

At block 201, an action is identified, in an unlocked state, based onsensor data collected by a sensor.

In one possible implementation, when in a working state, the terminalcontinuously collects sensor data by the sensor, and identifies theaction based on the sensor data. The sensor data may be collected by asingle sensor, or may be collected by multiple sensors of differenttypes. Correspondingly, the accuracy of identifying an action based onsensor data collected by multiple sensors is higher than the accuracy ofidentifying an action based on sensor data collected by a single sensor.

In some embodiments, in order to reduce the power consumption of theterminal and improve the security of the graphic code display, in thescreen unlocked state, the terminal identifies action based on thesensor data; in the screen locked state, the terminal stops identifyingaction.

The terminal may also continue to identify an action in the lockedstate, which is not limited in embodiments of the present disclosure.

In some embodiments, Sensorhub is configured to set an actionidentification algorithm for identifying an action, which is used as asoftware and hardware solution based on a low-power Micro ControllerUnit (MCU) and a lightweight Real Time Operating System (RTOS) forconnecting and processing sensor data collected by various sensors. Insome embodiments, a System on Chip (SoC) of the terminal is integratedwith a Micro Controller Unit (MCU), where an operating system runs onthe SoC, a Real Time Operating System (RTOS) runs on the MCU, and theMCU performs action identification based on the sensor data collected bythe sensor.

At block 202, an image is captured by a camera, in response to anidentified pre-action. The pre-action is an action prior to displayingthe graphic code to a code scanning device.

In the disclosure embodiment, the first level condition that triggersthe automatic display of the graphic code is the action condition. Theterminal detects whether the identified action belongs to thepre-action. If it does, it determines that the action condition issatisfied, and the terminal captures the image by the camera. If it doesnot, it determines that the action condition is not satisfied, and theterminal continues to identify the action (with no need to perform asubsequent process).

In the disclosure embodiment, the pre-action is not a touch action on aninterface element, i.e. it is not an action for the user interfacedisplayed on the terminal screen. The pre-action is an action to adjustthe orientation of the terminal screen based on the scanning directionof the code scanning device so that the graphic code displayed on theterminal screen is facing the scanning direction, rather than an actionto wake up the screen or unlock the terminal. In some embodiments, thepre-action is a gesture action.

The pre-action is a trigger action set by default on the terminal. Forexample, the pre-action is a gesture action before the user normallydisplays the graphic code to the code scanning device; or the pre-actionis a trigger action defined by the user, for example, the pre-action isa specific gesture action selected by the user from several candidateactions.

In one possible implementation, the terminal sends an interrupt to theSoC when it identifies that the user has performed the pre-actionthrough an action identification algorithm, and the SoC instructs thecamera to turn on and capture an image based on the interrupt.

As the graphic code displayed on the screen needs to be directly facingthe code scanning device, the terminal captures an image through thefront camera on one side of the screen; and in order to improve thesuccess rate of subsequent detection of the code scanning device, thecamera continuously captures multiple frames of images, or videoshooting.

At block 203, detection result is acquired, by detecting the codescanning device in the captured image.

When using the terminal, the user may perform an action similar to thepre-action without the need to display the graphic code. In thedisclosure embodiment, in order to reduce the false trigger probabilityof the graphic code, a second level condition that triggers theautomatic display of the graphic code is the device condition, and theterminal scans the captured image to detect the code scanning device andacquires the detection result. The detection result is used to indicatewhether a code scanning device exists directly in front of the terminalscreen. If the detection result indicates that there is a code scanningdevice, the device condition is determined to be satisfied and theautomatic display of the graphic code is triggered; if the detectionresult indicates that there is no code scanning device, the devicecondition is determined not to be satisfied and there is no need todisplay the graphic code.

In some embodiments, in addition to indicating whether there is a codescanning device, the detection result includes at least one of thedevice types of the identified code scanning device and the location ofthe code scanning device.

At block 204, the graphic code is displayed, in response to thedetection result indicating that the captured image contains the codescanning device.

When it is detected that there is a code scanning device directly infront of the terminal screen, the terminal automatically displays thegraphic code. The graphic code is dynamically determined according tothe current scene, or the graphic code is preset by the user (e.g., agraphic code that is commonly use or in a specific application).

In some embodiments, according to the form of the graphic code, thegraphic code may be a two-dimensional code or a barcode, etc.; accordingto the use of the graphic code, the graphic code may be a payment code,a membership code or a ride code, etc. The embodiment of the presentdisclosure does not limit the form and use of the graphic code.

In one possible embodiment, the terminal stops displaying the graphiccode after the display duration reaches a preset duration, and revertsto the original user interface (i.e., the user interface displayed onthe terminal before the execution of the pre-action). For example, thepreset duration may be 5 seconds.

Obviously, with the solution provided by this embodiment, the user onlyneeds to perform the pre-action, and point the terminal screen directlyto the code scanning device, and then the terminal automaticallydisplays the graphic code without cumbersome interface touch operations.Moreover, with the pre-action as a precondition, the terminal does notneed to turn on the camera for a long time to capture images, reducingthe overall power consumption of the terminal when the graphic codeautomatic display function is turned on.

In summary, in an implementation of this disclosure, the terminalidentifies whether the user performs the pre-action by identifying anaction on the sensor data collected by the sensor, and further capturesan image by the camera when the pre-action is identified, andautomatically displays the graphic code when the captured image isdetected to contain the code scanning device. In the entire graphic codedisplay process, the user only needs to perform the pre-action withoutpre-opening an application and the graphic code display function,simplifying the graphic code display process. Moreover, afteridentifying the pre-action, the terminal uses image identificationtechnology for detecting the code scanning device, which helps to reducethe probability of mis-display of the graphic code, and improve theaccuracy of the timing of the graphic code display and the security ofthe graphic code display.

In addition, in this embodiment, instead of continuously capturing animage by the camera in the unlocked state and detecting the codescanning devices in the captured images, the terminal determines thepresence of a scanning demand and captures the image and identifies thecode scanning device by setting the pre-action as a precondition fortriggering the image when the presence of a pre-action is identified,which helps to avoid the waste of processing resources caused bycapturing image and identifying the code scanning device, when there isno demand for code scanning, helps to reduce the power consumption ofthe terminal.

In some embodiments, the implementation process of capturing the imageby the camera, in response to an identified pre-action, is as follows.The method may include determining a flipping action as the pre-action,and capturing the image by a front camera, in response to identifyingthat the flipping action meets a flipping direction condition, whereinthe front camera is on one side of the screen.

In some embodiments, the implementation process of determining aflipping action as the pre-action and capturing an image by a frontcamera, in response to identifying that the flipping action meets aflipping direction condition is as follows. The method may includedetermining the flipping action as the pre-action, in response toidentifying the flipping action in a vertical state, the flipping actionindicating a change in screen orientation from a first direction to asecond direction, the first direction and the second direction beingrelative directions; or determining the flipping action as thepre-action, in response to identifying the flipping action that changesfrom a vertical state to a horizontal state, with the screen facingdownward in the horizontal state; or determining the flipping action asthe pre-action, in response to identifying a flipping action thatchanges from a vertical state to a horizontal state, with the screenfacing upward in the horizontal state.

In some embodiments, the implementation process of capturing an image bya camera, in response to an identified pre-action, is as follows. Themethod may include determining a shaking action as the pre-action, andcapturing an image by a front camera, the front camera being on one sideof screen, in response to the identified action being the shaking actionin a preset direction, and number of the shaking action being a presetnumber; or determining a knocking action as the pre-action, andcapturing an image by a front camera, the front camera being on one sideof screen, in response to the identified action being the knockingaction, and number of the knocking action being a preset number.

In some embodiments, the implementation process of identifying anaction, when in an unlocked state, based on sensor data collected by asensor, is as follows. The method may include acquiring N frames ofgravitational acceleration data continuously collected by anaccelerometer, each frame of the gravitational acceleration datacontaining an acceleration value in the three-axis direction, N being aninteger greater than or equal to 2; inputting the N frames ofgravitational acceleration data into an action identification model toacquire an action identification result output by the actionidentification model, the action identification model being aconvolutional neural network model.

In some embodiments, the implementation process of inputting the Nframes of gravitational acceleration data into an action identificationmodel to acquire an action identification result output by the actionidentification model is as follows. The method may include, in responseto a change of z-axis gravitational acceleration data from a first valueto a second value in the N frames of gravitational acceleration data,inputting the N frames of gravitational acceleration data into theaction identification model to acquire the action identification resultoutput by the action identification model; OR in response to a change ofz-axis gravitational acceleration data from a first value to a thirdvalue in the N frames of gravitational acceleration data and a change ofx-axis gravitational acceleration data from the third value to the firstvalue, inputting the N frames of gravitational acceleration data intothe action identification model to acquire the action identificationresult output by the action identification model; OR in response to achange of z-axis gravitational acceleration data from a first value to athird value in the N frames of gravitational acceleration data and achange of y-axis gravitational acceleration data from the third value tothe first value, inputting the N frames of gravitational accelerationdata into the action identification model to acquire the actionidentification result output by the action identification model; OR inresponse to a change of z-axis gravitational acceleration data from afourth value to a second value in the N frames of gravitationalacceleration data and a change of y-axis gravitational acceleration datafrom the fourth value to a third value, inputting the N frames ofgravitational acceleration data into the action identification model toacquire the action identification result output by the actionidentification model; OR in response to a change of x-axis gravitationalacceleration data from a third value to a first value in the N frames ofgravitational acceleration data and a change of y-axis gravitationalacceleration data from a fourth value to a third value, inputting the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result, output by the actionidentification model; OR in response to a change of z-axis gravitationalacceleration data from a fourth value to a third value in the N framesof the gravitational acceleration data, and a change of y-axisgravitational acceleration data from a fourth value to a first value,inputting the N frames of gravitational acceleration data into theaction identification model to acquire the action identification result,output by the action identification model; OR in response to a change ofz-axis gravitational acceleration data from a fourth value to a firstvalue in the N frames of the gravitational acceleration data, and achange of y-axis gravitational acceleration data from a fourth value toa third value, inputting the N frames of gravitational acceleration datainto the action identification model to acquire the actionidentification result output by the action identification model;wherein, the first value is greater than the fourth value, the fourthvalue is greater than the third value, the third value is greater thanthe second value, the first value and the fourth value are both positivevalues, the second value is a negative value.

In some embodiments, the implementation process of displaying thegraphic code is as follows. The method may include acquiring currentscene information, the current scene information including at least oneof geographic location information, time information, and recent codescanning information; determining a graphic code type based on thecurrent scene information; displaying graphic codes belonging to thegraphic code type.

In some embodiments, the implementation process of determining graphiccode type based on the current scene information is as follows. Themethod may include determining the graphic code type being a ride code,in response to the current scene information indicating a scene as acommuting scene, wherein, in the commuting scene, the geographiclocation information indicates a commuting ride station, and/or, thetime information indicates a commuting time period, and/or, the recentcode scanning information indicates a transportation inbound codeinformation; determining the graphic code type being a payment code, inresponse to the current scene information indicating a commodity paymentscene.

In some embodiments, the implementation process of displaying thegraphic code belonging to the graphic code type is as follows. Themethod may include in response to determining at least two of thegraphic code types based on the current scene information, determining atarget graphic code type from the at least two of the graphic codetypes, and determining a graphic code type other than the target graphiccode type as a candidate graphic code type; displaying the graphic codebelonging to the target graphic code type, and displaying a switchingcontrol corresponding to the candidate graphic code type; wherein afterthe displaying the graphic code belonging to the graphic code type, themethod further comprises: displaying the graphic codes belonging to thecandidate graphic code type, in response to a trigger operation on theswitching control.

In some embodiments, the implementation process of displaying thegraphic code, further is as follows. The method may include acquiring acode scanning device type contained in the detection result, anddetermining the graphic code type matching the code scanning devicetype; displaying the graphic code belonging to the graphic code type.

In some embodiments, the implementation process of displaying thegraphic code, further is as follows. The method may include determininga display position for the graphic code, based on an image position ofthe scanning device in the image, in response to the detection resultindicating the image containing the code scanning device; displaying thegraphic code at the display position.

In some embodiments, the implementation process of displaying thegraphic code, further is as follows. The method may include displayingthe graphic code, in response to the detection result indicating theimage containing the code scanning device, and a size of the codescanning device in the image being greater than a size threshold.

In some embodiments, the implementation process of detecting the codescanning device in the captured image to acquire detection result is asfollows. The method may include inputting the captured image into thecode scanning device detection model to acquire the detection resultoutput by the code scanning device detection model, wherein the codescanning device detection model being a mobile terminal convolutionalneural network model, and a convolutional layer of the code scanningdevice detection model using depth separable convolution, and the codescanning device detection model being jointly trained with softmax lossand focal loss.

In one possible implementation, in order to reduce the learning cost forthe user to use the automatic graphic code display function, theterminal sets the gesture action as the pre-action, which is the actionbefore the user normally displays the graphic code to the code scanningdevice, so that the user may automatically display the graphic code andcomplete the code scanning by performing the regular graphic codedisplay action without changing the user’s habit of displaying thegraphic code. On the basis of FIG. 2 , the block 202 may be replacedwith the block 202A as shown in FIG. 3 .

At block 202A, a flipping action is determined as the pre-action and theimage is captured by a front camera, in response to identifying that theflipping action meets a flipping direction condition. The front camerais on one side of the screen.

Usually, the user first displays the graphic code through a series ofinterface operations, and then flips the terminal to point the screendisplaying the graphic code to the code scanning device, so that thecode scanning device scans the graphic code.

Therefore, in the embodiment of the present disclosure, the terminalsets the flipping action before displaying the graphic code as thepre-action of the graphic code display.

Moreover, through the statistical analysis of the scanning mode of thecode scanning device, it is found that the scanning direction of thecode scanning device usually includes the forward, upward and downwarddirections. Correspondingly, when the scanning direction of the codescanning device is the forward direction, the user needs to flip theterminal so that the screen is located in front of the scanning device;when the code scanning direction of the scanning device is the upwarddirection, the user needs to flip the terminal so that the screen islocated above the code scanning device; and when the code scanningdirection of the code scanning device is the downward direction, theuser needs to flip the terminal so that the screen is located below thecode scanning device.

Therefore, in order to improve the identification accuracy of thepre-action, in this embodiment, the terminal determines the flippingaction belonging to the flipping direction condition as the pre-action.

For the above-mentioned three different scanning directions of the codescanning device, the terminal identifies the pre-action may include thefollowing possible implementations.

1. The terminal determines the flipping action as the pre-action, inresponse to identifying the flipping action in a vertical state and theflipping action indicating a change in screen orientation from a firstdirection to a second direction. The first direction and the seconddirection are relative directions.

When the scanning direction of the code scanning device is the forwarddirection, the regular scanning action of the user is to flip theterminal in the vertical state so that the screen of the terminal isflipped from facing the user to facing the code scanning device. Sincethe user is normally facing the code scanning device, the terminaldetermines that the flipping action belongs to the pre-action and theflipping action is used to display the graphic code to the code scanningdevice in the forward scanning direction, when the flipping action inthe vertical state is identified and the flipping action indicates achange in the screen orientation from the first direction to therelative second direction.

It should be noted that the above-mentioned vertical state is notabsolutely vertical. When the angle between the terminal screen and thevertical direction is less than a first angle threshold (e.g. 30°), theterminal is considered to be in a vertical state; when the angle betweenthe first direction and the second direction is greater than a secondangle threshold (e.g. 150°), the terminal determines that the firstdirection and the second direction are opposite directions.

For example, as shown in FIG. 4 , in the process of using the mobilephone 41, when the user needs to show the payment code to the codescanning device 42 in the forward scanning direction, the user onlyneeds to flip the mobile phone 41 so that the orientation of the screenchanges from facing the user to facing the code scanning device 42. Themobile phone 41 captures the forward image through the front camera onone side of the screen, and automatically displays the payment code tocomplete the payment when the forward image is detected to contain thecode scanning device 42.

2. The terminal determines the flipping action as the pre-action, inresponse to identifying the flipping action that changes from a verticalstate to a horizontal state, with the screen facing downward in thehorizontal state.

When the scanning direction of the code scanning device is the upwarddirection, the regular scanning action of the user is to flip theterminal from the vertical state to the horizontal state, so that thescreen of the terminal faces downward and is located above the codescanning device. Therefore, the terminal further detects the screenorientation in the horizontal state when the flipping action isidentified from the vertical state to the horizontal state. If thescreen faces downward, the terminal determines that the flipping actionbelongs to the pre-action and the flipping action is used to display thegraphic code to the code scanning device in the upward scanningdirection.

It should be noted that the above-mentioned vertical state is notabsolutely vertical. When the angle between the terminal screen and thevertical direction is less than a first angle threshold (e.g., 30°), theterminal is considered to be in a vertical state. And, theabove-mentioned horizontal state is not absolutely horizontal. When theangle between the terminal screen and the horizontal direction is lessthan a third angle threshold (e.g., 30°), the terminal is considered tobe in a horizontal state.

For example, as shown in FIG. 5 , in the process of using the mobilephone 41, when the user needs to show the payment code to the codescanning device 42 in the upward scanning direction, the user only needsto flip the mobile phone 41 so that the screen faces downwardhorizontally and towards the scanning area of the code scanning device.The mobile phone 41 captures the forward image through the front cameraon one side of the screen and automatically displays the payment code tocomplete the payment when the forward image is detected to contain thecode scanning device 42.

3. The terminal determines the flipping action as the pre-action, inresponse to identifying a flipping action that changes from a verticalstate to a horizontal state, with the screen facing upward in thehorizontal state.

When the scanning direction of the code scanning device is the downwarddirection, the regular scanning action of the user is to flip theterminal from the vertical state to the horizontal state, so that thescreen of the terminal faces upward and is located below the codescanning device. Therefore, the terminal further detects the screenorientation in the horizontal state when the flipping action isidentified from the vertical state to the horizontal state. If thescreen faces upward, the terminal determines that the flipping actionbelongs to the pre-action and the flipping action is used to display thegraphic code to the code scanning device in the downward scanningdirection.

It should be noted that the above-mentioned vertical state is notabsolutely vertical. When the angle between the terminal screen and thevertical direction is less than a first angle threshold (e.g., 30°), theterminal is considered to be in the vertical state. And theabove-mentioned horizontal state is not absolutely horizontal. When theangle between the terminal screen and the horizontal direction is lessthan a third angle threshold (e.g., 30°), the terminal is considered tobe in the horizontal state.

For example, as shown in FIG. 6 , in the process of using the mobilephone 41, when the user needs to show the payment code to the codescanning device 42 in the downward scanning direction, the user onlyneeds to flip the mobile phone 41 so that the screen faces upwardhorizontally and towards the scanning area of the code scanning device.The mobile phone 41 captures the forward image through the front cameraon one side of the screen, and automatically displays the payment codeto complete the payment when the forward image is detected to containthe code scanning device 42.

In this embodiment, the terminal achieves automatic display of thegraphic code by setting the flipping action in the line with the user’shabit of displaying the graphic code as the pre-action, without changingthe user’s existing habit of displaying the graphic code, which improvesthe efficiency of the graphic code display while reducing the learningcost of users for new functions.

In the daily use of the terminal, the user may perform an action similarto the graphic code display action without the graphic code displaydemand. For example, a handheld mobile phone is placed flat on thedesktop, and at this time if the phone triggers the camera to capture animage and detect the code scanning device, it will waste processingresources and increase the power consumption of the phone. In order toimprove the identification accuracy of the demand for graphic codedisplay, in another possible implementation, the terminal may set thegesture action different from the normal graphic code display as thepre-action. On the basis of FIG. 2 , the block 202 may be replaced withthe block 202B or 202C as shown in FIG. 7 .

At block 202B, a shaking action is determined as the pre-action and animage is captured by a front camera in response to the identified actionbeing the shaking action in a preset direction and number of the shakingaction being a preset number. The front camera is on one side of thescreen.

In one possible implementation, the user may set the pre-action ondemand when enabling the automatic display of the graphic code, whereinthe pre-action may be selected from candidate actions, or the pre-actionis customized by the user on the basis of the candidate actions, or thepre-action is completely user-defined.

In some embodiments, if the pre-action is user-defined, the terminalneeds to set the action identification algorithm corresponding to thepre-action in Sensorhub; if the pre-action is selected from thecandidate actions, the terminal needs to enable the actionidentification algorithm corresponding to the candidate actions inSensorhub.

In this embodiment, the pre-action may be a shaking action, wherein theshaking direction of the shaking action is a preset direction and thenumber of the shaking action is a preset number. For example, theshaking direction may be left and right shaking, up and down shaking,back and forth shaking (based on the vertical state), the number ofshaking action may be 2, 3, etc.

At block 202C, a knocking action is determined as the pre-action, and animage is captured by the front camera, in response to the identifiedaction being the knocking action and number of the knocking action beinga preset number. The front camera is on one side of the screen.

In addition to the shaking action, in this embodiment, the pre-actionmay be a knocking action, which may be a knocking action on the backcover of the terminal, or a knocking action on the screen or a knockingaction on any terminal edge frame. The number of knocking action is apreset number. For example, the pre-action may be two knocks on the backcover of the terminal.

It should be noted that this embodiment is only illustrated by shakingaction and knocking action. In other possible implementations, the usermay set other forms of the pre-action as required; this implementationdoes not constitute a limitation.

In addition, in order to further improve the identification accuracy ofthe demand for graphic code display, in one possible implementation, theterminal may set “shaking action + flipping action” or “knockingaction + flipping action” as the precondition for capturing image, whichis not repeated in this implementation.

The graphic code to be displayed by the user using the terminal may varyin different application scenes. For example, when buying a commodity,the user needs to use the terminal to display the payment code. Whentaking transportation, the user needs to use the terminal to display theride code. In order to improve the accuracy of automatic display of thegraphic code in different application scenes, in this disclosureembodiment, the terminal needs to determine the graphic code that meetsthe current application scene based on the application scene from atleast two graphic codes that are supported for display, and display thegraphic code. Exemplary embodiments are below for illustration.

FIG. 8 illustrates a flowchart of a graphic code display method providedin another exemplary embodiment of the present disclosure. The methodmay include the following.

At block 801, action is identified based on sensor data collected by asensor.

In one possible implementation, when the accelerometer set in theterminal has a three-axis gravitational acceleration data capturefunction, the terminal acquires N frames of gravitational accelerationdata continuously collected by the accelerometer, so as to input the Nframes of gravitational acceleration data into the action identificationmodel and acquire the action identification result output by the actionidentification model.

The accelerometer collects gravitational acceleration data at a samplingfrequency during operation. Each frame of gravitational accelerationdata contains acceleration values in the triaxial direction (i.e.,acceleration values in six directions: front, back, left, right, up anddown), wherein N is an integer greater than or equal to 2.

For example, the accelerometer collects gravitational acceleration datawith a sampling frequency of 100 Hz, i.e., 100 frames of gravitationalacceleration data are collected per second. The terminal takes 20 framesas a sampling window size and inputs the 20 consecutive frames ofgravitational acceleration data in the sampling window to the actionidentification model, which performs the identification action.

In one possible implementation, the action identification model is apre-trained convolutional neural network model. In the process oftraining the action identification model, an initial model is trainedwith positive samples of the sensor data collected by the sensor whenthe pre-action is met and negative samples of the sensor data collectedby the sensor when the pre-action is not met. Finally, the trainedaction identification model meets the action identification accuracyrequirement.

In the daily use of the terminal, the pre-action only accounts for avery small portion, and if the gravitational acceleration data iscontinuously input to the action identification model in the unlockedstate, then the action identification model will identify a large numberof invalid actions, resulting in a waste of processing resources andbringing additional power consumption to the terminal. In order toreduce the power consumption of the terminal while ensuring theidentification accuracy of the pre-action, in one possibleimplementation, the terminal filters the gravitational accelerationdata, filters out the filtered gravitational acceleration data thatobviously does not belong to the pre-action, and inputs thegravitational acceleration data (i.e., the gravitational accelerationdata that may belong to the pre-action) into the action identificationmodel for further identification.

In some embodiments, when the pre-action is the flipping action, theflipping action usually comprises at least one of the following:flipping the screen from upward to downward,

flipping the screen from upward to forward horizontal, flipping thescreen from upward to forward vertical, flipping the screen frombackward vertical to downward state,

flipping the screen from backward vertical to forward horizontal,flipping the screen from backward vertical to forward vertical, andflipping the screen from backward vertical to upward.

In one possible implementation, the terminal filters the gravitationalacceleration data and identifies action on the filtered gravitationalacceleration data by the action identification model including thefollowing.

1. The screen is flipped from upward to downward.

In response to a change of z-axis gravitational acceleration data from afirst value to a second value in the N frames of gravitationalacceleration data, the N frames of gravitational acceleration data areinputted into the action identification model to acquire the actionidentification result output by the action identification model.

FIG. 9 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from upward to downward. Thehorizontal axis indicates the number of frames of the sampled frames,the vertical axis indicates the value of the gravitational accelerationdata, the first curve 911 indicates the x-axis gravitationalacceleration data, the second curve 912 indicates the y-axisgravitational acceleration data, and the third curve 913 indicates thez-axis gravitational acceleration data.

As can be seen from FIG. 9 , in a first stage (frame 0 to frame 120),the terminal is in the stable state with the screen facing upward, thevalues of z-axis gravitational acceleration data are close to 9.8. In asecond stage (frame 120 to frame 200), the terminal is in the process offlipping the screen up and down, and the gravitational acceleration datain all three axes change substantially. In a third stage (frame 200 toframe 360), the terminal is in a stable state with the screen facingdownward, the values of z-axis gravitational acceleration data are closeto -9.8, while the values of the gravitational acceleration data onx-axis and y-axis are essentially the same as in the first stage.

Therefore, in one possible implementation, when the values of z-axisgravitational acceleration data in the N frames (e.g., 100 frames)gravitational acceleration data is detected to change from the firstvalue to the second value, the terminal determines that it may flip thescreen from upward state to downward state, and input the N framesgravitational acceleration data into the action identification model foraction identification. The first value is greater than the second valueand the first value is positive and the second value is negative; forexample, the first value is 9.8 and the second value is -9.8.

2. The screen is flipped from upward to forward horizontal.

In response to a change of z-axis gravitational acceleration data from afirst value to a third value in the N frames of the gravitationalacceleration data, and a change of x-axis gravitational accelerationdata from a third value to a first value, the N frames of gravitationalacceleration data are inputted into the action identification model toacquire the action identification result output by the actionidentification model.

FIG. 10 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from upward to forward horizontal.The horizontal axis indicates the number of frames of the sampledframes, the vertical axis indicates the value of the gravitationalacceleration data, the first curve 1011 indicates the x-axisgravitational acceleration data, the second curve 1012 indicates they-axis gravitational acceleration data, and the third curve 1013indicates the z-axis gravitational acceleration data.

It can be seen from FIG. 10 , in a first stage (frame 0 to frame 150),the terminal is in the stable state with the screen facing upward, thevalues of z-axis gravitational acceleration data are close to 9.8, andthe values of x-axis gravitational acceleration data are close to 0. Ina second stage (frame 150 to frame 230), the terminal is in the processof flipping the screen upward to forward horizontal, during the flippingprocess, the gravitational acceleration data in the x-axis, y-axis andz-axis change substantially. In a third stage (frame 230 to frame 360),the terminal is in the stable state with the screen facing forwardhorizontal, the values of z-axis gravitational acceleration data areclose to 0, while the values of the x-axis gravitational accelerationdata are close to 9.8.

Therefore, in one possible implementation, when it is detected that thevalues of the z-axis gravitational acceleration data in the N framesgravitational acceleration data change from the first value to the thirdvalue and the values of the x-axis gravitational acceleration datachange from the third value to the first value, the terminal determinesthat it may flip the screen from upward to forward horizontal, and inputthe N frames gravitational acceleration data into the actionidentification model for action identification. Wherein, the first valueis greater than the third value the first value is positive, forexample, the first value is 9.8 and the third value is 0.

3. The screen is flipped from upward to forward vertical.

In response to a change of z-axis gravitational acceleration data from afirst value to a third value in the N frames of the gravitationalacceleration data and a change of y-axis gravitational acceleration datafrom a third value to a first value, the N frames of gravitationalacceleration data are inputted into the action identification model toacquire the action identification result output by the actionidentification model.

FIG. 11 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from upward to forward vertical. Thehorizontal axis indicates the number of frames of the sampled frames,the vertical axis indicates the value of the gravitational accelerationdata, the first curve 1111 indicates the x-axis gravitationalacceleration data, the second curve 1112 indicates the y-axisgravitational acceleration data, and the third curve 1113 indicates thez-axis gravitational acceleration data.

It can be seen that in a first stage (frame 0 to frame 150), theterminal is in a stable state with the screen facing upward, the valuesof z-axis gravitational acceleration data are close to 9.8, the valuesof x-axis gravitational acceleration data are close to 0. In a secondstage (frame 150 to frame 230), the terminal is in the process offlipping the screen from upward to forward vertical, during the flippingprocess, the values of gravitational acceleration data in the three axesof x-axis, y-axis and z-axis are changing substantially. In a thirdstage (frame 230 to frame 360), the terminal is in the stable state withthe screen facing forward vertical, the values of z-axis gravitationalacceleration data are close to 0, while the values of the y-axisgravitational acceleration data are close to 9.8.

Therefore, in one possible implementation, when it is detected that thevalues of the z-axis gravitational acceleration data in the N framesgravitational acceleration data change from the first value to the thirdvalue and the values of the y-axis gravitational acceleration datachange from the third value to the first value, the terminal determinesthat it may flip the screen from upward to forward vertical, and inputthe N frames gravitational acceleration data into the actionidentification model for action identification. Wherein, the first valueis greater than the third value, the first value is positive, forexample, the first value is 9.8 and the third value is 0.

4. The screen is flipped from backward vertical to downward.

In response to a change of z-axis gravitational acceleration data from afourth value to a second value in the N frames of gravitationalacceleration data, and a change of y-axis gravitational accelerationdata from a fourth value to a third value, the N frames of gravitationalacceleration data are inputted into the action identification model toacquire the action identification result output by the actionidentification model.

FIG. 12 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from backward vertical to downward.The horizontal axis indicates the number of frames of the sampledframes, the vertical axis indicates the value of the gravitationalacceleration data, the first curve 1211 indicates the x-axisgravitational acceleration data, the second curve 1212 indicates they-axis gravitational acceleration data, and the third curve 1213indicates the z-axis gravitational acceleration data.

It can be seen that in a first stage (frame 0 to frame 100), theterminal is in the stable state with the screen facing backwardvertical, the values of y-axis gravitational acceleration data arenormally greater than 5, the values of z-axis gravitational accelerationdata fluctuate around 5. In a second stage (frame 100 to frame 280), theterminal is in the process of flipping the screen from backward verticalto downward, during the flipping process, the values of gravitationalacceleration data in the three axes of x-axis, y-axis and z-axis arechanging substantially. In a third stage (frame 280 to frame 360), theterminal is in the stable state with the screen facing downward, thevalues of z-axis gravitational acceleration data are close to -9.8,while the values of the y-axis gravitational acceleration data are closeto 0.

Therefore, in one possible implementation, when it is detected that thevalues of the z-axis gravitational acceleration data in the N framesgravitational acceleration data change from the fourth value to thesecond value and the values of the y-axis gravitational accelerationdata change from the fourth value to the third value, the terminaldetermines that it may flip the screen from backward vertical todownward, and input the N frames gravitational acceleration data intothe action identification model for action identification. Wherein, thefourth value is greater than the third value, the third value is greaterthan the second value, the fourth value is positive, the second value isnegative, for example, the fourth value is 5, the third value is 0, andthe second value is -9.8.

5. The screen is flipped from backward vertical to forward horizontal.

In response to a change of x-axis gravitational acceleration data from athird value to a first value in the N frames of the gravitationalacceleration data, and a change of y-axis gravitational accelerationdata from a fourth value to a third value, the N frames of gravitationalacceleration data are inputted into the action identification model toacquire the action identification result output by the actionidentification model.

FIG. 13 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from backward vertical to forwardhorizontal. The horizontal axis indicates the number of frames of thesampled frames, the vertical axis indicates the value of thegravitational acceleration data, the first curve 1311 indicates thex-axis gravitational acceleration data, the second curve 1312 indicatesthe y-axis gravitational acceleration data, and the third curve 1313indicates the z-axis gravitational acceleration data.

It can be seen that in a first stage (frame 0 to frame 100), theterminal is in a stable state with the screen facing backward vertical,the values of y-axis gravitational acceleration data are normallygreater than 5, the values of z-axis gravitational acceleration datafluctuate around 5. In a second stage (frame 100 to frame 280), theterminal is in the process of flipping the screen from backward verticalto forward horizontal, during the flipping process, the values ofgravitational acceleration data in the three axes of x-axis, y-axis andz-axis are changing substantially. In a third stage (frame 280 to frame360), the terminal is in the stable state with the screen facing forwardhorizontal, the values of x-axis gravitational acceleration data areclose to 9.8, while the values of the y-axis gravitational accelerationdata are close to 0.

Therefore, in one possible implementation, when it is detected that thevalues of the x-axis gravitational acceleration data in the N framesgravitational acceleration data change from the third value to the firstvalue and the values of the y-axis gravitational acceleration datachange from the fourth value to the third value, the terminal determinesthat it may flip the screen from backward vertical to forward horizontaland input the N frames gravitational acceleration data into the actionidentification model for action identification. Wherein, the first valueis greater than the fourth value, the fourth value is greater than thethird value, the first value and the fourth value are positive, forexample, the first value is 9.8, the fourth value is 5, and the thirdvalue is 0.

6. The screen is flipped from backward vertical to forward vertical.

In response to a change of z-axis gravitational acceleration data from afourth value to a third value in the N frames of the gravitationalacceleration data, and a change of y-axis gravitational accelerationdata from a fourth value to a first value, the N frames of gravitationalacceleration data are inputted into the action identification model toacquire the action identification result, output by the actionidentification model.

FIG. 14 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from backward vertical to forwardvertical. The horizontal axis indicates the number of frames of thesampled frames, the vertical axis indicates the value of thegravitational acceleration data, the first curve 1411 indicates thex-axis gravitational acceleration data, the second curve 1412 indicatesthe y-axis gravitational acceleration data, and the third curve 1413indicates the z-axis gravitational acceleration data.

It can be seen that in a first stage (frame 0 to frame 150), theterminal is in the stable state with the screen facing backwardvertical, the values of y-axis gravitational acceleration data arenormally greater than 5, the values of z-axis gravitational accelerationdata fluctuate around 5, and the values of x-axis gravitationalacceleration data tend to 0. In a second stage (frame 150 to frame 230),the terminal is in the process of flipping the screen from backwardvertical to forward vertical, during the flipping process, the values ofgravitational acceleration data in the two axes of x-axis and z-axis arechanging substantially. In a third stage (frame 230 to frame 360), theterminal is in the stable state with the screen facing forward vertical,the values of y-axis gravitational acceleration data are close to 9.8,while the values of the z-axis gravitational acceleration data are closeto 0.

Therefore, in one possible implementation, when it is detected that thevalues of the z-axis gravitational acceleration data in the N framesgravitational acceleration data change from the fourth value to thethird value and the values of the y-axis gravitational acceleration datachange from the fourth value to the first value, the terminal determinesthat it may flip the screen from backward vertical to forward verticaland input the N frames gravitational acceleration data into the actionidentification model for action identification. Wherein, the first valueis greater than the fourth value, the fourth value is greater than thethird value, the first value and the fourth value are positive, forexample, the first value is 9.8, the fourth value is 5, and the thirdvalue is 0.

7. The screen is flipped from backward vertical to upward.

In response to a change of z-axis gravitational acceleration data from afourth value to a first value in the N frames of the gravitationalacceleration data, and a change of y-axis gravitational accelerationdata from a fourth value to a third value, the N frames of gravitationalacceleration data are inputted into the action identification model toacquire the action identification result output by the actionidentification model.

FIG. 15 illustrates the change of gravitational acceleration data duringthe process of flipping the screen from backward vertical to upward. Thehorizontal axis indicates the number of frames of the sampled frames,the vertical axis indicates the value of the gravitational accelerationdata, the first curve 1511 indicates the x-axis gravitationalacceleration data, the second curve 1512 indicates the y-axisgravitational acceleration data, and the third curve 1513 indicates thez-axis gravitational acceleration data.

It can be seen that in a first stage (frame 0 to frame 150), theterminal is in the stable state with the screen facing backwardvertical, the values of y-axis gravitational acceleration data arenormally greater than 5, the values of z-axis gravitational accelerationdata fluctuate around 5, the values of x-axis gravitational accelerationdata tend to 0. In a second stage (frame 150 to frame 230), the terminalis in the process of flipping the screen from backward vertical toupward, during the flipping process, the values of gravitationalacceleration data in the two axes of y-axis and z-axis are changingsubstantially. In a third stage (frame 230 to frame 360), the terminalis in the stable state with the screen facing upward, the values ofy-axis gravitational acceleration data are close to 0, while the valuesof the z-axis gravitational acceleration data are close to 9.8.

Therefore, in one possible implementation, when it is detected that thevalues of the z-axis gravitational acceleration data in the N framesgravitational acceleration data change from the fourth value to thefirst value and the values of the y-axis gravitational acceleration datachange from the fourth value to the third value, the terminal determinesthat it may flip the screen from backward vertical to upward and inputthe N frames gravitational acceleration data into the actionidentification model for action identification. Wherein, the first valueis greater than the fourth value, the fourth value is greater than thethird value, the first value and the fourth value are positive, forexample, the first value is 9.8, the fourth value is 5, and the thirdvalue is 0.

It should be noted that in addition to the gravitational accelerationdata collected by the accelerometer, the terminal may also identifyaction based on the angular velocity data collected by the angularvelocity sensor in order to improve the accuracy of the actionidentification, which is not limited in this embodiment.

At block 802, an image is captured by the camera, in response toidentifying the pre-action of the graphic code display. The pre-actionis an action prior to displaying the graphic code to a code scanningdevice.

The implementation of this block may be referred to above-mentionedblocks 202, 202A, 202B, and 202C, and this embodiment will not berepeated here.

At block 803, the captured image is inputted into the code scanningdevice detection model to acquire the detection result output by thecode scanning device detection model.

In one possible implementation, the terminal is provided with apre-trained code scanning device detection model, the input of the codescanning device detection model is images, and the output of the codescanning device detection model is whether the image contains a codescanning device.

Since the code scanning device detection model needs to be deployed andrun in the terminal, in order to reduce the size of the model whileensuring the accuracy of the model detection and improve the modelinference speed, the code scanning device detection model in thisembodiment is a mobile convolutional neural network model, and theconvolutional layer of the code scanning device detection model usesdepth separable convolution, i.e., the convolutional layer uses aconvolution of the form depthwise+pointwise for convolutionaloperations.

For example, the code scanning device detection model is based onmobilenet-v3, some of the convolutional layers use deep separableconvolution to improve the model inference speed without reducingaccuracy.

In the process of training the code scanning device detection model,softmax loss and focal loss are used for joint training, which reducesthe impact of the imbalance between positive and negative samples onmodel training and further improve the quality of the model.

In addition, for the training of the code scanning device detectionmodel, before the model is deployed, the model may be furtherstreamlined by cutting the network, optimizing the network andquantizing the model to improve the model inference speed to meet thereal-time detection needs of the terminal.

Regarding the construction of samples during training, in one possibleimplementation, images of different types, different sizes (size of thecode scanning device in the image), different parts and differentenvironments (e.g., different brightness and darkness scenes) of thecode scanning device may be used as samples for training to improve therobustness as well as the detection accuracy of different models.

In some embodiments, the code scanning device detection model has thefunction of getting the position of the code scanning device in theimage and/or outputting the device type of the code scanning device, inaddition to the function of detecting whether the image contains thecode scanning device.

At block 804, the current scene information is acquired in response tothe detection result indicating that the image contains the codescanning device. The current scene information includes at least one ofgeographic location information, time information and recent codescanning information.

When the captured image contains the code scanning device, the terminalfurther acquires the current scene information indicating the currentlylocated scene, wherein the current scene information containsinformation of different dimensions to improve the accuracy of thesubsequently determined scene.

In this embodiment, the current scene information includes thegeographic location information indicating the current location, thetime information indicating the current moment, and the recent codescanning information indicating the type of the graphic code displayedat the recent code scan.

In some embodiments, the geographic location information is determinedby the terminal based on at least one of the base station information,network connection information (such as WiFi connection information) orpositioning information (such as GPS information), and the geographiclocation information may be the latitude and longitude coordinates, orthe name of location. The present disclosure embodiment does not limitthe way and the presentation form of acquiring the geographic locationinformation.

In some embodiments, the time information may be the current time or thetime period of which the current time belongs (such as commuting timeperiod, working time period).

In some embodiments, the recent code scanning information includes thetype of the graphic code displayed at the recent code scan, the momentof the recent code scan and the geographic location information at therecent code scan, etc., which is not limited in this embodiment.

In an illustrative example, the current scene information acquired bythe terminal includes the following: convenience store XX (thegeographic location information), lunchtime (the time information) andscan code to exit Station B (the recent code scanning information).

At block 805, a graphic code type is determined based on the currentscene information.

Further, the terminal identifies the current application scene based onthe acquired current scene information and determines the graphic codetype matching the current application scene.

In one possible implementation, the terminal divides the applicationscene into a commuting scene and a commodity payment scene. Accordingly,the terminal determines the graphic code type to be a ride code inresponse to the scene indicated by the current scene information being acommuting scene; the terminal determines the graphic code type to be apayment code in response to the scene indicated by the current sceneinformation being a commodity payment scene.

In some embodiments, the terminal determines that the currentapplication scene is a commuting scene when the geographic locationinformation indicates that it is located at a commuting station, and/or,when the time information indicates that it is at a commuting timeperiod, and/or when the recent code scanning information indicates thatit is a transportation inbound code scan information.

In an illustrative example, the terminal determines that the currentapplication scene is the commuting scene when the current sceneinformation acquired by the terminal including Station B (the geographiclocation information), commuting time period (the time information) andscanning code to enter Station A (the recent code scanning information).

In some embodiments, the terminal determines that the current scene is acommodity payment scene, if the current scene indicates that the currentapplication scene is not the commuting scene.

In an illustrative example, the current scene information acquired bythe terminal includes the following: convenience store XX (thegeographic location information), lunchtime (the time information) andscanning code to exit Station B (the recent code scanning information).

It should be noted that this embodiment is only illustrative with thecommuting scene and the commodity payment scene, in other possibleimplementations, the scenes may be further divided according to actualneeds and set corresponding payment code types for different scenes,which is not limited in embodiments of the present disclosure.

At block 806, the graphic code belonging to the graphic code type isdisplayed.

In one possible implementation, when a unique graphic code type isdetermined based on the current scene information, the terminal displaysthe graphic code belonging to the graphic code type.

Schematically, as shown in FIG. 16 , when the mobile phone 1601determines that the user is located at an underground station based onthe acquired current scene information, the mobile phone 1601 displaysan underground ride code 1603 when the user flips the mobile phone 1601and places the screen squarely against the code scanning device 1602 ofthe underground gate.

As shown in FIG. 17 , when the mobile phone 1701 determines that theuser is located in a convenience store based on the acquired currentscene information, the mobile phone 1701 displays a payment code 1703when the user flips the mobile phone 1701 and places the screen squarelyagainst the code scanning device 1702 of the vendor.

In some embodiments, when there are at least two graphic codes belongingto the graphic code type, the terminal displays the most frequently usedgraphic code. For example, if the graphical code type is the paymentcode type and the graphic codes belonging to the payment code typeinclude a payment code 1 and a payment code 2, the terminal will displaythe more frequently used payment code 1.

In this embodiment, based on the current scene information acquired, theterminal determines the current application scene and the type ofgraphic code matching the current application scene, and then displaysthe graphic code belonging to the graphic code type, avoiding the userfrom switching the automatically displayed graphic code and improvingthe accuracy of the automatic display of graphic codes in differentscenes.

Normally, the size of the code scanning device in the captured image isrelatively large due to the close distance (e.g., around 10 cm) betweenthe terminal displaying the graphic code and the code scanning device.In order to avoid mis-display of the graphic code due tomisidentification of the remote code scanning device by the terminalwhen there is no demand for graphic code display. In one possibleimplementation, in response to the detection result indicating the imagecontains the code scanning device, the terminal further acquires thesize of the code scanning device in the image, and displays the graphiccode when the size is larger than the size threshold, and does notdisplay the graphic code when the size is smaller than the sizethreshold. For example, the size threshold is 200 px * 200 px.

In some embodiments, the size of the code scanning device in the imagemay be determined based on the bounding box of the code scanning devicein the detection result.

In some special situations, for example, when the user is buying acommodity at a convenience store located in an underground station, theterminal cannot accurately determine the current application scene basedon the current scene information. In such situation, in order to improvethe efficiency of the user in switching graphic code (the initialdisplay of the graphic code does not meet the actual needs of the user),in one possible implementation, the terminal provides a shortcutswitching control in the interface for the user to use. On the basis ofFIG. 8 , as shown in FIG. 18 , the above block 806 may be replaced bythe block 806A and the block 806B as follows; the block 807 may followthe block 806B.

At block 806A, in response to determining at least two of the graphiccode types based on the current scene information, a target graphic codetype is determined from the at least two of the graphic code types, andthe graphic code type other than the target graphic code type isdetermined as a candidate graphic code type.

In some embodiments, the target graphic code type may be the morefrequently used graphic code type of at least two graphic code types, orthe graphic code type that matches the application scene more closely,or a random graphic code type of at least two graphic code types.

For example, when the geographic location information acquired includesstation A and convenience store XX (convenience store XX is located instation A), the terminal cannot directly determine whether the userneeds to scan the code to enter station A or to scan the code to pay, atthis time, the target graphic code type may be determined as the paymentcode and the candidate graphic code type may be determined as the ridecode.

At block 806B, the graphic code belonging to the target graphic codetype and a switching control corresponding to the candidate graphic codetype are displayed.

In some embodiments, when the target graphic code needs to be displayed,the terminal gives priority to the graphic code belonging to the targetgraphic code type. In order to improve the efficiency of the user inswitching the graphic code, the terminal displays the graphic code whiledisplaying switching control corresponding to the candidate graphic codetype in the user interface, and the switching control is used to triggera switch to display the graphic code belonging to the candidate graphiccode type.

The number of the switching controls corresponds to the number of thecandidate graphic code types, i.e., different candidate graphic codetypes correspond to different switching controls.

Exemplarily, as shown in FIG. 19 , the terminal determines that thetarget graphic code type is the payment code and the candidate graphiccode type is the ride code. When the graphic code is displayed, theterminal displays the payment code 1901 in the user interface anddisplays the switching control 1902 corresponding to the ride code.

At block 807, the graphic code belonging to the candidate graphic codetype is displayed, in response to a trigger operation on the switchingcontrol.

When the displayed graphic code does not meet the actual demand of theuser, the user may switch the graphic code through the switchingcontrol. Accordingly, when a trigger operation is triggered on theswitching control, the terminal switches to display the graphic codebelonging to the candidate graphic code type without the need for theuser to manually open the application corresponding to the candidategraphic code type and enter the graphic code display entry.

Exemplarily, as shown in FIG. 19 , when it is necessary to use theterminal to scan the ride code for entering the underground station, theuser may trigger the terminal to switch to display the ride code 1903 byclicking on the switching control 1902.

In this embodiment, when the application scene cannot be accuratelyjudged based on the current scene information, the terminal displays theswitching control while displaying the graphic code, so that the usermay quickly switch to the graphic code to be displayed through theswitching control, simplifying the operation process of switching thegraphic code and improving the efficiency of switching the graphic code.

In other possible implementations, when the detection result containsthe code scanning device type, the terminal may further determine thegraphic code type that matches the code scanning device type and displaythe graphic code belonging to the graphic code type, improving theaccuracy of the graphic code display.

For example, when the detection result indicates that the code scanningdevice in the image is an underground gate scanner, the terminaldetermines that the graphic code type is the ride code; when thedetection result indicates that the code scanning device in the image isa vendor scanning gun, the terminal determines that the graphic codetype is the payment code.

In some embodiments, when training the code scanning device detectionmodel for detecting code scanning device, the training sample includes acode scanning device type label. During the training of the model, themodel parameters are adjusted using the code scanning device type labelas supervision so that the predicted code scanning device type output ofthe model tends to that code scanning device type label.

In the related graphic code display process, since the user may observethe display position of the graphic code in the screen in advance, therelative position between the terminal screen and the code scanningdevice may be adjusted based on the display position to improve the codescanning success rate of the code scanning device. In the presentdisclosure embodiment, the user cannot observe the display position ofthe graphic code before the terminal displays the graphic code, so itmay affect the success rate of the code scanning.

In order to improve the success rate of the code scanning, in onepossible implementation, when the detection result indicates that theimage contains the code scanning device, the terminal further determinesthe image position of the code scanning device in the image, so as todetermine the graphic code display position based on the image positionof the code scanning device in the image, and then displays the graphiccode at the graphic code display position. By adjusting the graphic codedisplay position in the interface, it achieves an effect similar to thatof manually adjusting the relative orientation of the screen by the userand improves the success rate of the code scanning.

When the image position of the code scanning device in the image isdeflected to the upper part (compared to the image position of the codescanning device for the standard graphic code display action), theterminal shifts the graphic code display position upward (based on thegraphic code display position for the standard graphic code displayaction). When the image position of the code scanning device in theimage is deflected to the lower part, the terminal shifts the graphiccode display position downward. When the image position of the codescanning device in the image is deflected to the left part, the terminalshifts the display position of the graphic code to the left. When theimage position of the code scanning device in the image is deflected tothe right part, the terminal shifts the display position of the graphiccode to the right.

In this embodiment, the terminal dynamically adjusts the graphic codedisplay position according to the image position of the code scanningdevice in the image, achieving a similar effect to the user manuallyadjusting the relative orientation of the screen and improving thesuccess rate of the code scanning device in the automatic graphic codedisplay scheme.

FIG. 20 illustrates a structural diagram of a graphic code displaydevice according to an embodiment of the disclosure. The device may beimplemented as all or part of a terminal by software, hardware or acombination of both. The device comprises: an action identificationmodule 2001, configured to identify an action, in an unlocked state,based on sensor data collected by a sensor; an image capturing module2002, configured to capture an image by a camera, in response to anidentified pre-action, the pre-action being the action prior todisplaying the graphic code to a code scanning device; a devicedetection module 2003, configured to detect the code scanning device inthe captured image to acquire detection result; a graphic code displaymodule 2004, configured to display a graphic code, in response to thedetection result indicating that the image contains the code scanningdevice.

In some embodiments, the image capturing module 2002 is configured todetermine a flipping action as the pre-action and capture image by frontcamera, in response to identifying the flipping action meets flippingdirection condition, wherein the front camera is on one side of thescreen.

In some embodiments, the image capturing module 2002 is configured todetermine the flipping action as the pre-action, in response toidentifying the flipping action in a vertical state and the flippingaction indicating a change in screen orientation from a first directionto a second direction, the first direction and the second directionbeing relative directions; OR determine the flipping action as thepre-action, in response to identifying the flipping action that changesfrom a vertical state to a horizontal state, with the screen facingdownward in the horizontal state; OR determine the flipping action asthe pre-action, in response to identifying a flipping action thatchanges from a vertical state to a horizontal state, with the screenfacing upward in the horizontal state.

In some embodiments, the image capturing module 2002 is configured todetermine a shaking action as the pre-action and capture image by frontcamera, the front camera being on one side of screen, in response to theidentified being the shaking action in a preset direction, and number ofthe shaking action being a preset number; OR determine a knocking actionas the pre-action and capture image by front camera, the front camerabeing on one side of screen, in response to the identified action beingthe knocking action and number of the knocking action being a presetnumber.

In some embodiments, the action identification module 2001 comprises: adata acquiring unit, configured to acquire N frames of gravitationalacceleration data, continuously collected by the accelerometer, eachframe of the gravitational acceleration data containing an accelerationvalue in the three-axis direction, and N being an integer greater thanor equal to 2.

In some embodiments, an identification unit is configured to input the Nframes of gravitational acceleration data into an action identificationmodel to acquire an action identification result output by the actionidentification model, the action identification model being aconvolutional neural network model.

In some embodiments, the identification unit is configured to inresponse to a change of z-axis gravitational acceleration data from afirst value to a second value in the N frames of the gravitationalacceleration data, input the N frames of gravitational acceleration datainto the action identification model to acquire the actionidentification result, output by the action identification model; or inresponse to a change of z-axis gravitational acceleration data from afirst value to a third value in the N frames of gravitationalacceleration data and a change of x-axis gravitational acceleration datafrom the third value to the first value, input the N frames ofgravitational acceleration data into the action identification model toacquire the action identification result output by the actionidentification model; or in response to a change of z-axis gravitationalacceleration data from a first value to a third value in the N frames ofgravitational acceleration data and a change of y-axis gravitationalacceleration data from the third value to the first value, input the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result output by the actionidentification model; or in response to a change of z-axis gravitationalacceleration data from a fourth value to a second value in the N framesof gravitational acceleration data and a change of y-axis gravitationalacceleration data from the fourth value to the third value, input the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result output by the actionidentification model; or in response to a change of x-axis gravitationalacceleration data from a third value to a first value in the N frames ofgravitational acceleration data and a change of y-axis gravitationalacceleration data from the fourth value to the third value, input the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result output by the actionidentification model; or in response to a change of z-axis gravitationalacceleration data from a fourth value to a third value in the N framesof gravitational acceleration data and a change of y-axis gravitationalacceleration data from the fourth value to the first value, input the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result output by the actionidentification model; or in response to a change of z-axis gravitationalacceleration data from a fourth value to a first value in the N framesof gravitational acceleration data and a change of y-axis gravitationalacceleration data from the fourth value to the third value, input the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result output by the actionidentification model; wherein, the first value is greater than thefourth value, the fourth value is greater than the third value, thethird value is greater than the second value, the first value and thefourth value is a positive value, and the second value is a negativevalue.

In some embodiments, the graphic code display module 2004 comprises: ascene information acquisition unit, configured to acquire current sceneinformation, the current scene information includes at least one ofgeographic location information, time information, and recent codescanning information.

A first type determination unit, configured to determine a graphic codetype based on the current scene information.

A graphic code display unit, configured to display the graphic codebelonging to the graphic code type.

In some embodiments, the first type determination unit is configured todetermine the graphic code type being a ride code, in response to thecurrent scene information indicating a scene as a commuting scene,wherein, in the commuting scene, the geographic location informationindicates a commuting ride station, and/or, the time informationindicates a commuting time period, and/or, the recent code scanninginformation indicates a transportation inbound code scan information;determine the graphic code type being a payment code, in response to thecurrent scene information indicating a commodity payment scene.

In some embodiments, the graphic code display unit is configured to inresponse to determining at least two of graphic code types based on thecurrent scene information, determine a target graphic code type from theat least two of the graphic code types and determine a graphic code typeother than the target graphic code type as a candidate graphic codetype; display the graphic code belonging to the target graphic code typeand display a switching control corresponding to the candidate graphiccode type.

The device further comprises: a switch display module, configured todisplay the graphic code belonging to the candidate graphic code type,in response to a trigger operation on the switching control.

In some embodiments, the graphic code display module 2004 comprises: afirst type determination unit, configured to acquire a code scanningdevice type contained in the detection result, and determine the graphiccode type matching the code scanning device type; display the graphiccode belonging to the graphic code type; a graphic code display unit,configured to display the graphic code belonging to the graphic codetype.

In some embodiments, the graphic code display module 2004 furthercomprises: a location determination unit, configured to determine adisplay position for the graphic code, based on an image position of thecode scanning device in the image, in response to the detection resultindicating the image containing the code scanning device; a graphic codedisplay unit, configured to display the graphic code at the graphic codedisplay position.

In some embodiments, the graphic code display module 2004 is furtherconfigured to display the graphic code, in response to the detectionresult indicating the image containing the code scanning device and asize of the code scanning device in the image being greater than a sizethreshold.

In some embodiments, the device detection module 2003 is configured toinput the captured image into the code scanning device detection modelto acquire the detection result output by the code scanning devicedetection model, wherein the code scanning device detection model beinga mobile convolutional neural network model, convolutional layer of thecode scanning device detection model using depth separable convolution,and the code scanning device detection model being jointly trained withsoftmax loss and focal loss.

In summary, in an implementation of this disclosure, the terminalidentifies whether the user performs the pre-action by identifyingaction on the sensor data collected by the sensor, and further capturesimage by the camera when the pre-action is identified; and automaticallydisplays the graphic code when the captured image is detected to containthe code scanning device. In the entire graphic code display process,the user only needs to perform the pre-action without pre-opening anapplication and the graphic code display function, simplifying thegraphic code display process. Moreover, after the pre-action isidentified, the image identification technology is used to detect thecode scanning device, which helps to reduce the probability ofmis-display of the graphic code, and improve the accuracy of the timingof the graphic code display and the security of the graphic codedisplay.

In the above-mentioned device embodiment, the process of implementingthe functions of the individual modules or units may be referred to themethod embodiment above. This embodiment will not be repeated here.

FIG. 21 illustrates a structural diagram of a terminal according to anexemplary embodiment of the disclosure. The terminal 2100 may be asmartphone, a tablet computer, a wearable device, etc. The terminal 2100in the present disclosure may comprise one or more of the followingcomponents: processor 2110, memory 2120, display screen 2130 and camera2140.

The processor 2110 may include one or more processing cores. Theprocessor 2110 takes various interfaces and wires to connect variousparts within the entire electronic device. The processor 2110 mayperform various functions and process data of the electronic device 100by running or executing instructions, programs, code sets, orinstruction sets stored in the memory 2120, and by invoking data storedin the memory 2120. In some embodiments, the processor 2110 may beachieved in the hardware form by taking at least one of a digital signalprocessing (DSP), a field-programmable gate array (FPGA), a programmablelogic array (PLA). The processor 2110 may integrate one or a combinationof: a central processing unit (CPU), a graphics processing unit (GPU), aneural-network processing unit (NPU) and a modem. The CPU maysubstantially process the operating system, the user interface,applications, and so on; the GPU may be configured to render and drawthe displayed content on the display screen 2130; the NPU may beconfigured to implement Artificial Intelligence (AI) functions; themodem may be configured for wireless communication. It shall beunderstood that the modem may not be integrated into the processor 2110,and may be configured as an independent communication chip.

The memory 2120 may include a random-access memory (RAM), or a read-onlymemory (ROM). In some embodiments, the memory 2120 may include anon-transitory computer-readable storage medium. The memory 2120 may beconfigured to store instructions, programs, codes, code sets orinstruction sets. The memory 2120 may include a program storage area anda data storage area. The program storage area may store instructions forimplementing an operating system, instructions for implementing at leastone function (such as a touch function, a sound playing function, animage displaying function, and so on), instructions for implementingeach of the following method embodiments, and so on. The data storagearea may store data created while the terminal 2100 is being used, suchas audio data, phone books, and so on.

The display screen 2130 is a display component for displaying images.The display screen 2130 may be designed as a full screen, curved screen,shaped screen, double-sided screen, or foldable screen, which are notlimited by the embodiments. In addition to having a display function,the display screen 2130 may also have a touch function, i.e. the displayscreen 2130 is a touch display screen.

The camera 2140 is a component for capturing images. In someembodiments, the camera 2140 may be an RGB camera or a depth camera. Inthis disclosure embodiment, the display screen 2130 is provided with acamera 2140 (i.e., a front camera) on one side of the display screen2130, which is used to capture images on the side facing the displayscreen 2130.

In addition, it will be understood by those skilled in the art that thestructure of the terminal 2100 illustrated in the accompanying drawingsabove does not constitute a limitation of the terminal 2100, and thatthe terminal 2100 may include more or fewer components than illustrated,or a combination of certain components, or a different arrangement ofcomponents. For example, the terminal 2100 may further includecomponents such as radio-frequency circuitry, a sensor, an audiocircuit, a Wireless Fidelity (Wi-Fi) component, a power supply, aBluetooth component, and other components, which are not describedherein.

Embodiments of the present disclosure further provide a computerreadable medium having at least one instruction stored thereon, the atleast one instruction being loaded and executed by the processor toimplement the graphic code display method as described in the aboveembodiments.

Embodiments of the present disclosure provide a computer program productor computer program, the computer program product or computer programincluding computer instructions, the computer instructions being storedin a computer-readable storage medium. A processor of the terminal readsthe computer instructions from the computer-readable storage medium, andthe processor executes the computer instructions such that the terminalperforms the graphic code display method provided in the above aspect.

Those skilled in the art should be aware that in one or more of theabove examples, the functions described in the embodiments of thepresent disclosure may be implemented with hardware, software, firmware,or any combination thereof. When implemented using software, thesefunctions may be stored in a computer-readable medium or transmitted asone or more instructions or code on a computer-readable medium. Thecomputer-readable medium includes computer storage medium andcommunication medium, where the communication medium includes any mediumthat facilitates the transmission of computer programs from one place toanother. The storage medium may be any available medium accessible to ageneral purpose or specialized computer.

The descriptions above are merely optional embodiments of the presentdisclosure, but are not intended to limit the present disclosure. Anymodifications, equivalent replacements, or improvements made within thespirit and principle of the present disclosure should fall within theprotection scope of the present disclosure.

Other features and aspects of the disclosed features will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, which illustrate, by way of example, thefeatures in accordance with embodiments of the disclosure. The summaryis not intended to limit the scope of any embodiments described herein.

What is claimed is:
 1. A graphic code display method, comprising:identifying an action, in an unlocked state, based on sensor datacollected by a sensor; capturing an image by a camera in response to anidentified pre-action, wherein the pre-action is the action prior todisplaying the graphic code to a code scanning device; detecting thecode scanning device in the captured image to acquire a detectionresult; and displaying the graphic code, in response to the detectionresult indicating that the image contains the code scanning device. 2.The method according to claim 1, wherein the step of capturing an imageby a camera, in response to an identified pre-action, comprises:determining a flipping action as the pre-action and capturing the imageby a front camera, in response to identifying that the flipping actionmeets a flipping direction condition, wherein the front camera is on oneside of a screen.
 3. The method according to claim 2, wherein the stepof determining a flipping action as the pre-action and capturing theimage by the front camera, in response to identifying that the flippingaction meets the flipping direction condition, comprises: determiningthe flipping action as the pre-action, in response to identifying theflipping action in a vertical state, the flipping action indicating achange in screen orientation from a first direction to a seconddirection, the first direction and the second direction being relativedirections; OR determining the flipping action as the pre-action, inresponse to identifying the flipping action that changes from thevertical state to a horizontal state, with the screen facing downward inthe horizontal state; OR determining the flipping action as thepre-action, in response to identifying the flipping action that changesfrom the vertical state to the horizontal state, with the screen facingupward in the horizontal state.
 4. The method according to claim 1,wherein the step of capturing an image by a camera, in response to anidentified pre-action, comprises: determining a shaking action as thepre-action and capturing the image by a front camera, the front camerabeing on one side of a screen, in response to the identified actionbeing the shaking action in a preset direction and a number of theshaking action being a preset number; OR determining a knocking actionas the pre-action and capturing the image by the front camera, the frontcamera being on one side of the screen, in response to the identifiedaction being the knocking action and a number of the knocking actionbeing a preset number.
 5. The method according to claim 1, wherein thestep of identifying an action, in an unlocked state, based on sensordata collected by a sensor comprises: acquiring N frames ofgravitational acceleration data continuously collected by anaccelerometer, each frame of the gravitational acceleration datacontaining an acceleration value in a three-axis direction, N being aninteger greater than or equal to 2; inputting the N frames ofgravitational acceleration data into an action identification model toacquire an action identification result output by the actionidentification model, the action identification model being aconvolutional neural network model.
 6. The method according to claim 5,wherein the step of inputting the N frames of gravitational accelerationdata into an action identification model to acquire an actionidentification result output by the action identification modelcomprises: in response to a change of z-axis gravitational accelerationdata from a first value to a second value in the N frames ofgravitational acceleration data, inputting the N frames of gravitationalacceleration data into the action identification model to acquire theaction identification result output by the action identification model;OR in response to a change of z-axis gravitational acceleration datafrom a first value to a third value in the N frames of gravitationalacceleration data and a change of x-axis gravitational acceleration datafrom the third value to the first value, inputting the N frames ofgravitational acceleration data into the action identification model toacquire the action identification result output by the actionidentification model; OR in response to a change of z-axis gravitationalacceleration data from a first value to a third value in the N frames ofgravitational acceleration data and a change of y-axis gravitationalacceleration data from the third value to the first value, inputting theN frames of gravitational acceleration data into the actionidentification model to acquire the action identification result outputby the action identification model; OR in response to a change of z-axisgravitational acceleration data from a fourth value to a second value inthe N frames of gravitational acceleration data and a change of y-axisgravitational acceleration data from the fourth value to a third value,inputting the N frames of gravitational acceleration data into theaction identification model to acquire the action identification resultoutput by the action identification model; OR in response to a change ofx-axis gravitational acceleration data from a third value to a firstvalue in the N frames of gravitational acceleration data and a change ofy-axis gravitational acceleration data from a fourth value to a thirdvalue, inputting the N frames of gravitational acceleration data intothe action identification model to acquire the action identificationresult, output by the action identification model; OR in response to achange of z-axis gravitational acceleration data from a fourth value toa third value in the N frames of the gravitational acceleration data,and a change of y-axis gravitational acceleration data from a fourthvalue to a first value, inputting the N frames of gravitationalacceleration data into the action identification model to acquire theaction identification result, output by the action identification model;OR in response to a change of z-axis gravitational acceleration datafrom a fourth value to a first value in the N frames of thegravitational acceleration data, and a change of y-axis gravitationalacceleration data from a fourth value to a third value, inputting the Nframes of gravitational acceleration data into the action identificationmodel to acquire the action identification result output by the actionidentification model; wherein, the first value is greater than thefourth value, the fourth value is greater than the third value, thethird value is greater than the second value, the first value and thefourth value are both positive values, and the second value is anegative value.
 7. The method according to claim 1, wherein the step ofdisplaying the graphic code comprises: acquiring current sceneinformation, the current scene information including at least one ofgeographic location information, time information, and recent codescanning information; determining a graphic code type based on thecurrent scene information; displaying the graphic code belonging to thegraphic code type.
 8. The method according to claim 7, wherein the stepof determining the graphic code type based on the current sceneinformation comprises: determining the graphic code type being a ridecode, in response to the current scene information indicating a scene asa commuting scene, wherein in the commuting scene, the geographiclocation information indicates a commuting ride station, and/or, thetime information indicates a commuting time period, and/or, the recentcode scanning information indicates a transportation inbound code scaninformation; determining the graphic code type being a payment code, inresponse to the current scene information indicating a commodity paymentscene.
 9. The method according to claim 7, wherein the step ofdisplaying the graphic code belonging to the graphic code typecomprises: in response to determining at least two of graphic code typesbased on the current scene information, determining a target graphiccode type from the at least two of the graphic code types anddetermining a graphic code type other than the target graphic code typeas a candidate graphic code type; displaying the graphic code belongingto the target graphic code type and displaying a switching controlcorresponding to the candidate graphic code type; wherein after thedisplaying the graphic code belonging to the graphic code type, themethod further comprises: displaying the graphic code belonging to thecandidate graphic code type, in response to a trigger operation on theswitching control.
 10. The method according to claim 1, wherein the stepof displaying the graphic code further comprises: acquiring a codescanning device type contained in the detection result, and determiningthe graphic code type matching the code scanning device type; displayingthe graphic code belonging to the graphic code type.
 11. The methodaccording to claim 1, wherein the step of displaying the graphic codefurther comprises: determining a display position for the graphic code,based on an image position of the code scanning device in the image, inresponse to the detection result indicating the image containing thecode scanning device; displaying the graphic code at the displayposition.
 12. The method according to claim 1, wherein the step ofdisplaying the graphic code further comprises: displaying the graphiccode, in response to the detection result indicating the imagecontaining the code scanning device and a size of the code scanningdevice in the image being greater than a size threshold.
 13. The methodaccording to claim 1, wherein the step of detecting the code scanningdevice in the captured image to acquire detection result furthercomprises: inputting the captured image into the code scanning devicedetection model to acquire the detection result output by the codescanning device detection model, the code scanning device detectionmodel being a mobile convolutional neural network model, a convolutionallayer of the code scanning device detection model using depth separableconvolution, the code scanning device detection model being jointlytrained with softmax loss and focal loss.
 14. A terminal, comprising: aprocessor and a memory configured to store instructions which, whenexecuted by the processor, cause the one or more processors to: identifyan action, in an unlocked state, based on sensor data collected by asensor; capture an image by a camera in response to an identifiedpre-action, wherein the pre-action is the action prior to displaying thegraphic code to a code scanning device; detect the code scanning devicein the captured image to acquire a detection result; and display thegraphic code, in response to the detection result indicating that theimage contains the code scanning device.
 15. The terminal of claim 14,wherein the instructions that cause the processor to capture an image bya camera, in response to an identified pre-action that, when executed,cause the processor to: determine a flipping action as the pre-actionand capturing the image by a front camera, in response to identifyingthat the flipping action meets a flipping direction condition, whereinthe front camera is on one side of a screen.
 16. The terminal of claim15, wherein the instructions that cause the processor to determine aflipping action as the pre-action and capturing the image by the frontcamera, in response to identifying that the flipping action meets theflipping direction condition that, when executed, caused the processorto: determine the flipping action as the pre-action, in response toidentifying the flipping action in a vertical state, the flipping actionindicating a change in screen orientation from a first direction to asecond direction, the first direction and the second direction beingrelative directions; OR determine the flipping action as the pre-action,in response to identifying the flipping action that changes from thevertical state to a horizontal state, with the screen facing downward inthe horizontal state; OR determine the flipping action as thepre-action, in response to identifying the flipping action that changesfrom the vertical state to the horizontal state, with the screen facingupward in the horizontal state.
 17. The terminal of claim 14, whereinthe instructions that cause the processor to capture an image by acamera, in response to an identified pre-action that, when executed,cause the processor to: determine a shaking action as the pre-action andcapturing the image by a front camera, the front camera being on oneside of a screen, in response to the identified action being the shakingaction in a preset direction and a number of the shaking action being apreset number; OR determine a knocking action as the pre-action andcapturing the image by the front camera, the front camera being on oneside of the screen, in response to the identified action being theknocking action and a number of the knocking action being a presetnumber.
 18. A non-transitory computer readable storage medium havingstored instructions that is executed by a processor of a terminal, causethe processor of a terminal to: identify an action, in an unlockedstate, based on sensor data collected by a sensor; capture an image by acamera in response to an identified pre-action, wherein the pre-actionis the action prior to displaying the graphic code to a code scanningdevice; detect the code scanning device in the captured image to acquirea detection result; and display the graphic code, in response to thedetection result indicating that the image contains the code scanningdevice.
 19. The non-transitory computer-readable medium of claim 18,wherein the instructions that cause the processor to capture an image bya camera, in response to an identified pre-action that, when executed,cause the processor to: determine a flipping action as the pre-actionand capturing the image by a front camera, in response to identifyingthat the flipping action meets a flipping direction condition, whereinthe front camera is on one side of a screen.
 20. The non-transitorycomputer-readable medium of claim 19, wherein the instructions thatcause the processor to determine a flipping action as the pre-action andcapturing the image by the front camera, in response to identifying thatthe flipping action meets the flipping direction condition that, whenexecuted, cause the processor to: determine the flipping action as thepre-action, in response to identifying the flipping action in a verticalstate, the flipping action indicating a change in screen orientationfrom a first direction to a second direction, the first direction andthe second direction being relative directions; OR determine theflipping action as the pre-action, in response to identifying theflipping action that changes from the vertical state to a horizontalstate, with the screen facing downward in the horizontal state; ORdetermine the flipping action as the pre-action, in response toidentifying the flipping action that changes from the vertical state tothe horizontal state, with the screen facing upward in the horizontalstate.